Modeling Skill Combination Patterns for Deeper Knowledge Tracing

This paper explores the problem of modeling student knowledge in complex learning activities where multiple skills are required at the same time, such as in the programming domain. In such cases, it is not clear how the evidence of student performance translates to individual skills. As a result, traditional approaches to knowledge modeling, such as Knowledge Tracing (KT), which traces students’ knowledge of each decomposed individual skill, might fall short. We argue that skill combinations might carry extra specific knowledge, and mastery should be asserted only when a student can fluently apply skills in combination with other skills in different contexts. We propose a data-driven framework to model skill combination patterns for tracing students’ deeper knowledge. We automatically identify significant skill combinations from data and construct a conjunctive knowledge model with a hierarchical skill representation based on a Bayesian Network. We also propose a novel evaluation framework primarily focuses on the knowledge inference quality, since we argue that traditional prediction metrics no longer suffice to differentiate between shallow and deep knowledge modeling. Our experiments on datasets collected from two programming learning systems show that proposed model significantly increases mastery inference accuracy and tends to more reasonably distribute students’ efforts comparing with traditional KT models and its nonhierarchical counterparts. Our work serves as a first step towards building skill application context sensitive model for modeling students’ deep, robust learning.